A coherent memory system in a computer typically includes a high-capacity off-chip external memory coupled with a relatively smaller internal on-chip memory cache. The cache mirrors a portion of the data stored in the off-chip memory. A cache controller directs read requests of addresses of the external memory to the cache when the cache has copies of the data corresponding with those addresses. A read request to access the cache can be serviced with a lower access time than a read request to access the off-chip external memory.
In a coherent memory system with an external memory and a cache, the external memory and the cache remain coherent. In such a coherent memory system, the data stored in the cache either matches the copy of the data at an address in the external memory or is an updated version of the data for that address. Writes to external memory addresses are first written to the cache. Later, a hardware mechanism copies the data from the cache to the external memory under certain conditions.
A typical computer system generally tightly couples the cache with the external memory within a single memory sub-system. Upon any read request, the cache controller determines if the cache is storing the data at the requested address of the external memory. If the cache is storing the data, the memory sub-system reads the data directly from the cache. If the cache is not storing the data, the request is forwarded to the external memory.
The cache stores recently accessed data or likely-to-be accessed data in a plurality of “cache lines,” which are minimum units of storage within the cache. However, storing data in the cache that is eventually not used or is used infrequently is inefficient and detrimental to overall system performance because the cache capacity is significantly smaller than the capacity of the external memory. The memory sub-system passes a requested read address to the cache controller. If the controller indicates data for the external memory that is present in the cache, the version of the data in the cache is returned. If the data for the requested external memory address is not in the cache, a scenario known as a “cache miss,” the memory sub-system directs the read request to the external memory. In the “cache miss” scenario, the memory subsystem copies large quantities of sequential data bytes to a “cache line” within the cache. Typical cache systems anticipate that a read request to a particular external memory address is followed by a read request to the next higher external memory address. Caching data for sequential memory addresses allows subsequent sequential read requests to the memory sub-system to access data stored in the faster cache instead of requiring additional read requests to access data in the external memory.